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Abstract 

A general setting to study a certain type of formulas, expressing characters of 
the symmetric group & n explicitly in terms of descent sets of combinatorial 
objects, has been developed by two of the authors. This theory is further 
investigated in this paper and extended to the hyperoctahedral group B n . 
Key ingredients are a new formula for the irreducible characters of B n , the 
signed quasisymmetric functions introduced by Poirier, and a new family of 
matrices of Walsh-Hadamard type. Applications include formulas for natural 
Reactions on coinvariant and exterior algebras and on the top homology of 
a certain poset in terms of the combinatorics of various classes of signed 
permutations, as well as a R„-analogue of an equidistribution theorem of 
Desarmenien and Wachs. 
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1. Introduction 

One of the main goals of combinatorial representation theory, as described 
in the survey article [0], is to provide explicit formulas which express the 
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values of characters of interesting representations as weighted enumerations 
of nice combinatorial objects. Perhaps the best known example of such a 
formula is the Murnaghan-Nakayama rule [291 page 117] [431 Section 7.17] 
for the irreducible characters of the symmetric group © n . 

Several such formulas of a more specific type, expressing characters of the 
symmetric groups and their Iwahori-Hecke algebras in terms of the distribu¬ 
tion of the descent set over classes of permutations, or other combinatorial 
objects, have been discovered in the past two decades. The prototypical 
example is Roichman’s rule [35] for the irreducible characters of & n (and 
the corresponding Hecke algebra), where the enumerated objects are either 
Knuth classes of permutations, or standard Young tableaux. Other notable 
examples include the character of the Gelfand model (i.e., the multiplicity- 
free sum of all irreducible characters) [3], characters of homogenous compo¬ 
nents of the coinvariant algebra |2], Lie characters [24], characters of Specht 
modules of zigzag shapes [23], characters induced from a lower ranked exte¬ 
rior algebra d! and fc-root enumerators [37], determined by the distribution 
of the descent set over involutions, elements of fixed Coxeter length, con- 
jugacy classes, inverse descent classes, arc permutations and fc-roots of the 
identity permutation, respectively. The formulas in question evaluate these 
characters by { — 1,0, l}-weighted enumerations of the corresponding classes 
of permutations, where exactly the same weight function appears in all sum¬ 
mations. Some new examples are given in Sections [7]-[8] of this paper. 

An abstract framework for this phenomenon, which captures all afore¬ 
mentioned examples, was proposed in 0. Characters which are expressed 
by such formulas (see Definition 3.1) are called fine characters , and classes 
which carry them are called fine sets. It was shown in [U]j that the equal¬ 
ity of two fine characters is equivalent to the equidistribution of the descent 
set over the corresponding fine sets. This implies, in particular, the equiv¬ 
alence of classical theorems of Lusztig-Stanley in invariant theory m and 
Foata-Schiitzenberger in permutation statistics |21j . Furthermore, it was 
shown in [B] that fine sets can be characterized by the symmetry and Schur- 
positivity of the associated quasisymmetric functions. For the latter, the 
reader is referred to Gessel-Reutenauer’s seminal paper m 

This paper investigates this setting further and provides a nontrivial ex¬ 
tension to the hyperoctahedral group B n . Section [3T] gives a more explicit 
version of the main result of [6]. This version (Theorem 3.2) states that a 
given ©^-character is carried by a fine set B if and only if its Frobenius char¬ 
acteristic is equal to the quasisymmetric generating function of the descent 
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set over B. To extend this result to the group B n , suitable signed analogues 
of the concepts of fine characters and fine sets have to be introduced (see Def¬ 
inition 3.5) and suitable signed analogues of the fundamental quasisymmetric 

are employed. 


functions, namely those defined and studied by Poirier 
For the former task, a signed analogue of the concept of descent set is used 
(see Section 2.2) and an analogue of Roichman’s rule for the irreducible char¬ 
acters of B n (Theorem 4.1) is proven. The weights involved in this rule are 
used to define type B fine sets as those sets whose weighted enumeration 
determines a non-virtual B n - character. The main result (Theorem 3.6) of 
this paper states that a given £>„-character is carried by a fine set B if and 
only if its Frobenius characteristic is equal to the quasisymmetric generating 
function of the signed descent set over B. 

The proof of one direction of Theorem T6 amounts to the invertibility of a 
new family of matrices of Walsh-Hadamard type, introduced in Section 4.2| 
This is shown by a tricky computation of their determinants. Equidistri- 
bution and Schur-positivity phenomena, similar to those in [Bj, are easily 


derived from Theorem 3.6 
Sections 


(see Corollary 3.7). 


show that nearly all examples of fine ©^-characters men¬ 
tioned earlier have natural R ri -analogues. These include the irreducible char¬ 
acters of B n , the Gelfand model, the characters of the natural R n -action on 
the homogeneous components of the coinvariant algebra of type B, signed 
analogues of the Lie characters, characters induced from exterior algebras 
and fc-root enumerators, with corresponding fine sets consisting of elements 
of Knuth classes of type B n , involutions, signed permutations of fixed flag- 
inversion number or flag-major index, conjugacy classes in B n) signed ana¬ 
logues of arc permutations and /c-roots of the identity signed permutation, 
respectively. Theorem 3.6 implies explicit formulas for these characters in 
terms of the distribution of the signed descent set over the corresponding 
fine sets. Section ns is concerned with the character of the © n -action on 
the top homology of the poset of injective words, studied by Reiner and 
Webb [S], and with its natural R n -analogue. This allows us to interpret a 
theorem of Desarmenien and Wachs jT51 ITE] on the equidistribution of the 
descent set over derangements and desarrangements in & n in the language 
of © n -fme sets, and to derive a R n -analogue. The latter task provided much 
of the motivation behind this paper. 

The structure of this paper is as follows. Section [2] reviews background 
material from combinatorial representation theory and sets up the notation 
on permutations, partitions, Young tableaux, compositions, descent sets and 
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their signed analogues which is necessary to define fine sets. The main defi¬ 
nitions and results are stated in Section [3j which also derives the aforemen¬ 
tioned result on © n -fine sets (Theorem 3.2) from the main result of |6j. Sec¬ 
tion [f] states and proves the R„-analogue of Roichman’s rule (Theorem 4.1| ) 
and uses this result, together with methods from linear algebra, to prove the 
main result (Theorem 3.6) on R n -fine sets. As already mentioned, Sections[5]- 
[8] discuss applications and examples. Section [9] concludes with remarks and 
open problems. 


2. Background and notation 

This section fixes notation and briefly reviews background material re¬ 
garding the combinatorics of (signed) permutations and Young (bi)tableaux, 
the representation theory of the symmetric and hyperoctahedral groups and 
the theory of symmetric and quasisymmetric functions which will be needed 
in the sequel. More information on these topics and any undefined terminol¬ 
ogy can be found in [29], Chapter I] [43], Chapter 7] [04] [ 45] . 

Throughout this paper, x = (aq, X 2 , ...) and y = ( 2 / 1 , 2 / 2 , - - -) will be 
sequences of pairwise commuting indeterminates. For positive integers n we 
set [n] := (1,2,..., n} and := (1,2,..., n} U {I, 2,..., n}. The absolute 
value |a| of a G is the element of [n] obtained from a by simply forgetting 
the bar, if present. We will consider the set [n] totally ordered by its natural 
order < and Q n totally ordered by the order 

1 < r 2 < r ■■■ < r n < r 1 < r 2 < r ■■■ < r n, (2.1) 

corresponding to the right lexicographic order on the set [n] x { — ,+}, and 
endowed with the standard involution —)■ Q n , mapping a to a and vice 

versa, for every a G [n\. 

2.1. Compositions and partitions 

A composition of a positive integer n, written as « N n, is a sequence 
a = (aq, a 2 ,..., aq) of positive integers (parts) summing to n. We will write 
S(a ) = (ri, r 2 ,..., rq} for the set of partial sums r* = aq + a 2 + • ■ ■ + a* 
(1 < i < k) of such a, and will denote by Cornp(n) the set of all compositions 
of n. Clearly, the map which assigns 5(a) to a is a bijection from Cornp(n) 
to the set of all subsets of [n] which contain n. A partition of n, written as 
A b n, is a composition A of n whose parts appear in a weakly decreasing 
order. We shall also consider the empty composition (and partition) of n — 0. 
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A bipartition of n, written as (A, //) b n, is a pair (A, /i) of (possibly 
empty) partitions such that Ah k and /i h n — k for some 0 < k < n. 

A signed composition of n is a composition 7 of n some of the parts of 
which may be barred. Such a 7 can be encoded by its set of partial sums 
S'(y) = {si, s 2 ,..., Sfc}, where s\ < s 2 < ■ ■ ■ < Sk — n, together with the sign 
map e : S ( 7 ) —> { —, +} dehned by setting e(sj) = + if the ith part of 7 is 
unbarred and £(s;) = — otherwise. Clearly, the map which assigns the pair 
(S'(y), e) to 7 is a bijection from the set Comp s (n) of all signed compositions 
of n to the set consisting of all pairs (S', e), where S' is a subset of [n] 

which contains n and e : S' —>■ { —,+} is a map. Thus, as observed for 
instance in [251 Section 3.2], the total number of signed compositions of n is 
equal to 2 ■ 3 n_1 . We will consider the elements of S B (n), which we think of 
as “signed sets”, instead of the corresponding signed compositions, whenever 
this is notationally convenient. 

Given a = (S', e) G S B (u), with S' = {si, s 2 ,..., Sk} and Si < s 2 < • • • < 
Sfc = n, we may extend e to a map e : [n] —» { —, +} by setting i(j) := e(sj) 
for all Sj_i < j < Si (1 < i < k), where so := 0. We will refer to £ as the sign 
vector of a (and of the corresponding signed composition). The following 
notation will be useful in Section 12.41 

Definition 2.1. For a = (S',e) G £ B (n) we denote by wDes(a) the set of 
elements s t G S', other than Sk = n, for which either e(sj) = e(sj+i), or else 
e(sj) = + and e(s i+ i) = -. 

For example, if n = 9 and 7 = (2,1,2,1,3), then for the signed set 
a = (S', er) corresponding to 7 we have S' = { 2 ,3,5, 6 ,9}, e(2) = e( 6 ) = 
e(9) = +, e(3) = e(5) = —, wDes(cr) = {2,3,6} and sign vector £ = 

(+) +) — ) — ) — 1 +) +> +) +)■ 


2.2. Permutations and tableaux 

We will denote by © n the symmetric group of all permutations of the 
set [n], namely bijective maps w : [n] —> [n], and by SYT(A) the set of all 
standard Young tableaux of shape Ah n. We recall that the Robinson- 
Schensted correspondence is a bijection of fundamental importance from & n 
to the set of pairs (P, Q) of standard Young tableaux of the same shape and 
size n. The descent set of a permutation w G & n is dehned as Des(w) := 
{i G [n — 1] : w(i) > w{i + 1)}. The descent set Des(Q) of a standard 
Young tableau Q G SYT(A), where A b n, is the set of all i G [n — 1] for 
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which i + 1 appears in a lower row in Q than i does. A basic property of 
the Robinson-Schensted correspondence asserts that Des(w) = Des(Q(tc)), 
where (P(w),Q(w)) is the pair of tableaux associated to w E & n . 

The hyperoctahedral group B n consists of all signed permutations of 
length n, meaning bijective maps w : —)■ Q n such that w(a ) = b => 

w(a ) = b for every a E f2 n . Unless indicated otherwise, we will think of a 
signed permutation w E B n as the sequence of values (ta(l), w(2), ..., w(n)), 
namely as a permutation in & n , written in one line notation, with some of 
its entries barred. The (unsigned) descent set of w E B n is defined as the 
descent set of the sequence (w(l),w(2),... ,w(n )) with respect to the total 


order (2.1), namely Des(w) := {i E [n — 1] : w(i) > r w(i + 1)}. 


Definition 2.2. The signed (or colored ) descent set of w E B n , denoted 
sDes(w), is the signed set (S', e) E Y B {n) defined as follows: 

• The set S consists of n along with all s E [n — 1] for which either 
w(s ) > r w(s + 1) or w(s) is barred and w(s + 1) is unbarred. 


For every s E S, e(s) = — if w(s) is barred and e(s) = + otherwise. 


In other words, denoting |w| = (|iy(l)|,..., |iu(n)|) G 6 n and defining 
e w : [n] -E { —, +} by e w (i) = — if w(i) is barred and e w (i) = + otherwise, the 
set S is the union of Des(|u>|), {n} and {i E [n — 1] : e w (i) ^ £^(* + 1)}, while 
£ is the restriction of e w to S. For example, if iu = (5, 2, 8 ,1,3, 9,4, 6 , 7) G B 9 
then sDes(w) = (S,e) with S = {1,3, 6 , 7, 9} and e(l) = e(3) = £(9) = —, 
£( 6 ) = £(7) = +. We note that if a = sDes(w) for some w E B n then 
wDes(a) = Des(tc) := {i E [n — 1] : w(i) > r w{i + 1)}, where wDes(cr) is as 
in Definition 12.11 

Given a bipartition (A, (i) of n, a standard Young bitableau of shape (A, /i) 
and size n is a pair Q = ( Q + , Q~) of tableaux which are (strictly) increasing 
along rows and columns and have the following properties: (a) Q + has shape 
A; (b) Q~ has shape //; and (c) every element of [n] appears (exactly once) in 
either Q + or Q~. We will denote by SYT(A, //) the set of all standard Young 
bitableaux of shape (A,/i). 

The descent set , denoted by Des(Q), of a bitableau Q = ( Q + ,Q ~) G 
SYT(A, /i) consists of all i E [n — 1], such that i and * + 1 appear in the same 
tableau ( Q + or Q ~) and i + 1 appears in a lower row than i, or i appears in 
Q + and i + 1 appears in Q~. 
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Definition 2.3. The signed (or colored ) descent set , denoted by sDes(Q), of 
a bitableau Q = ( Q + ,Q~ ) € SYT(A , fi) is the element (S, e) E £ B (n) defined 
as follows: 

• The set S consists of n along with all s E [n — 1] for which either s and 
s + 1 appear in the same tableau ( Q + or Q~ ) and s + 1 appears in a 
lower row than s, or s and s + 1 appear in different tableaux. 

• For every s E S, e(s) = + if s appears in Q + and e(s) = — otherwise. 

Clearly, for the corresponding sign vector e : [n] —> {—,+} we have, for 
every i E [n], e{i) — + if i appears in Q + and e{i) — — if i appears in Q~. 

The Robinson-Schensted correspondence has a natural P n -analogue; see, 
for instance, [^1, Section 6 ] @1] and Section [5] This analogue is a bijection 
from B n to the set of pairs (P, Q ) of standard Young bitableaux of the same 
shape and size n. It has the property that sDes(w) = sDes(Q B (tc)), where 
(P B (ui),Q B (w)) is the pair associated to w E B n . 

2.3. Characters and symmetric functions 

Conjugacy classes in & n consist of permutations of given cycle type A h 
n. As a result, the (complex) irreducible characters of & n are indexed by 
partitions of n. We will denote by the irreducible character corresponding 
to A b n. For a class function x ■ —> C and composition a |= n, we will 

write x( a ) f° r tl ie value of x on an y element of & n whose cycle type is (the 
decreasing rearrangement of) a. 

We will denote by A(x) the C-algebra of symmetric functions in x. The 
algebra A(:r) encodes the combinatorics of characters of the symmetric groups 
via the Frobenius characteristic map, defined by 

ch(x) = V A :Mp w (x) (2.2) 

n\ 

6 „ 

where x '■ —> C is a class function, p w (x) = p a {x) for every permutation 

w E & n of cycle type a F n, and p a (x) is a power sum symmetric function. 
This map is a C-linear isomorphism from the space of such class functions on 
& n to the degree n homogeneous part A n (x) of A(x), and satisfies ch(y A ) = 
s \(x) for every A F n; see [431, Section 7.18] for a detailed discussion and 
further information. 
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The values y A (a) of the irreducible characters of & n appear in the Frobe- 
nius formula 

Pa(x) = ^x A (a)s A (x) (2.3) 


Ahn 


for & n , expressing p a (x) as a linear combination of Schnr functions. 

We now recall an explicit formula for y A (a). Let a = («i, a 2 ,..., a*,) G 


Comp(n) be a composition of n with S(a) = {ri, r 2 ,..., r*,}, as in Section 2.1 
(so that rk = n), and set r 0 = 0. A subset S of [n — 1] is said to be a-unimodal 
(or unimodal with respect to a) P, Definition 3.1] if the intersection of S with 
each of the sets {r , *_ 1 +1 ,..., iq — 1} (1 < i < k) is a prefix (possibly empty) of 
the latter. For instance, if a = (n), then the a-unimodal subsets of [n —1] are 
those equal to \p) for some p G {0,1,..., n — 1}; and if a = (1,1,..., 1), then 
every subset of [n — 1] is a-unimodal. As another example, if a = (3,1,4, 2), 
then {1,3, 5, 6} is a-unimodal but {1,3,5, 7} is not. We will denote by U a 
the set of a-unimodal subsets of [n — 1]. 

The following theorem is a special case of [351 Theorem 4]; see also [3 
Section 1.2] and references therein. A direct combinatorial proof appeared 


m 


Theorem 2.4. 


For all partitions Ahn and compositions a |= n, 


x \ a ) = J2 (— i) l Des ( < 3) x *-’( Q )l _ (2.4) 

QeSYT(A) 

Des(Q)eJ7o, 


The concept of a-unimodality, and Theorem 2.4 in particular, were re¬ 
cently applied to prove conjectures of Regev concerning induced charac¬ 
ters PH Section 9] and of Shareshian and Wachs concerning chromatic qua- 
sisymmetric functions [8j. 


We now briefly describe the analogue of this theory for the group B n . 
Conjugacy classes in B n) and hence (complex) irreducible R n -characters, are 
in one-to-one correspondence with bipartitions of n. More precisely, each 
element w G B n , viewed as a permutation of the set Q n , can be written as a 
product of disjoint cycles of total length 2 n. Moreover, if c = (ai, a 2 , ■ ■ ■, a*) 
is such a cycle then so is c = (ai, a 2 ,..., a*,), and either c and c are disjoint 
or else c = c. In the former case, the product cc is said to be a positive cycle 
of w, of length fc; otherwise k is necessarily even and c = c is said to be a 
negative cycle of w, of length k/2. Then the pair of partitions (a, /3) for which 




the parts of a (respectively, (3) are the lengths of the positive (respectively, 
negative) cycles of w is a bipartition of n, called the signed cycle type of w. 
Two elements of B n are congngate if and only if they have the same signed 
cycle type. We will denote by C a ,p the conjngacy class of elements of B n of 
signed cycle type (a, (3), by y A,/i the irreducible i?„-character corresponding 
to (A, /i) h n, and by y (a, (3) the value of a class function y : B n —> C at an 
arbitrary element of C a 


For A 

— (Ai, A 2 ,.. ■, A r ) h n, 

we set 


Pa 1 

[x,y) = ptpl 2 ---pl r 

and 

p>. y) = Pi, Pi, ■ ■ -f'i. 

where 

pi = + 

i> 1 

and 

pi = E (*? -»?) 

i> 1 


for k > 1. Thus p^(x,y) and pf(x,y) are homogeneous elements of degree n 
of A(x) ® A (y). The Frobenius characteristic of a class function y : B n —> C 
is defined as 

ch (x) = Y X(w)p w (x,y) (2.5) 

w&B n 

where p w (x,y) := p+(x, y)pp(x, y) if w G B n has signed cycle type (a, f3). 
The Frobenius characteristic is a C-linear isomorphism from the space of such 
class functions on B n to the degree n homogeneous part of A(x) <8) A (y). It 
satisfies ch(y'W) = s\(x)s IJ ,(y) for every bipartition (A,//) h n. The following 
lemma will be used in Section 17.11 

Lemma 2.5. ( [321 Lemma 21 (i)]) For the trivial character of B n we have 
ch(l B J = s n (x). 

The Frobenius formula for B n states that 

Pt( x ,y)Pp(x,y) = (x)s fl {y) (2.6) 

(A,/i)h n 


for every bipartition (a, (3) b n. 

An analogue of Theorem 2.4 for B n will be proved in Section [4] (see 
Theorem 4.1). 

We write (•, •) for the inner product on A(x) (respectively, on A(x) ® 

A (y)) for which the Schur functions s\(x) (respectively, the functions s\(x)s^(y)) 
form an orthonormal basis. 
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2-4- Quasisymmetric functions 

We denote by QSym” the C-vector space of homogeneous quasisymmet¬ 
ric functions of degree n in x. The fundamental quasisymmetric function 
associated to S' C [n — 1] is defined as 

K,s( x ) = Y x h x i2 ■ ■ ■ x in . (2.7) 

j£S => ij<ij +1 


The set {F nj> s(:r) : S' C [n — 1]} is known to be a basis of QSym”. The follow¬ 
ing proposition expresses the Schur function S\(x) as a linear combination of 
the elements of this basis. 

Proposition 2.6. ([121 Theorem 7.19.7]) For every Ah n, 

5a(*^) ^ F n,Des(Q) ( x ) • (2.8) 

QeSYT(A) 

Different type B analogues of quasisymmetric functions have been sug¬ 
gested mm- The 7? n -analogues of the fundamental quasisymmetric func¬ 
tions that we need were introduced (in the more general setting of r-colored 
permutations) by Poirier [32, Section 3] and were further studied in [TOl [25]. 
For a signed set a = (S', e) G E B (n) define 

F A X ,V) = Y z ii z i 2 -'- z in ( 2 -9) 

il<i2<-..<in 

j£ wDes(cr) => ij<ij + i 

where = x j. if ij = +, and z t] = y tj if ij = — (we use the notation F a , 
instead of F na , since n is determined by a = (S', e) as the largest element 
of S). For example, if n — 6 and S = {2,3,5, 6 } with sign vector e = 
(+, +, —, —, —, +), then 

?/) ^ ^ x i\ x I2y i^y i^y i§ X - 

We leave it to the reader to verify that this definition is equivalent to the 
one in [32]. The functions F a (x,y), where a ranges over all signed sets in 
E s (n), are linearly independent over C [221 Corollary 9] and, in fact, form a 
C-basis of a natural signed analogue of QSym”; see m Section 3.2], 
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3. Main results: Character formulas and fine sets 

This section introduces the concept of fine set for the hyperoctahedral 
group and explains the main results of this paper, to be proved in the fol¬ 
lowing section. The corresponding concepts and results from |5jj for the 
symmetric group will first be reviewed. 


3.1. Review of results for the symmetric group 


The following key definition from [SJ uses the notion of unimodality with 


respect to a composition, defined in Section 2.3 


Definition 3.1. (J6J Definition 1.3]) Let y be a character of the symmetric 
group & n . A set B , endowed with a map Des : B —» 2l n_1 l, is said to be a 
fine set for y if 

y (a) = ^(_i)|D-WxS(a)| (3.1.) 

beB a 

for every composition a of n, where B a is the set of all elements b G B for 
which Des(fc) is a-unimodal. 


For example, according to Theorem 2.4, the set SYT(A), endowed with 
the usual descent map Des : SYT(A) —> 2^ n ~ l \ is a fine set for the irreducible 
character y A . Several other examples (including some new ones) appear in 
Sections [5H8] 

The following theorem is a version of the main result of [6]; we include a 
proof since this version is only implicit in [6]. 


Theorem 3.2. Let y be a character of the symmetric group & n and B be a 
set endowed with a map Des : B —> 2^ n ~ l f Then B is a fine set for y if and 
only if 

ch (y) = ^F n ,De S (b)(x). (3.2) 

beB 

In particular, the distribution of the descent set over B is uniquely determined 
by y. 

Proof. Let us express y = Yl\\-n c aX A as a linear combination of irreducible 
characters. Using Proposition |2.6| we find that 

ch(y) = e A ch(y A ) = ^c A s A (a;) = c A ^ F n ,n eH(Q) (x). 

Ahn Ah n Ahn QeSYT(A) 
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From this we conclude that ( |3.2[ ) holds if and only if 

\{b G B : Des(fe) = S}\ = c A • \{Q e SYT(A) : Des(g) = 5)1 (3.3) 

Ah n 


we get 


for every S C [n — 1]. Assuming (3.2) holds and using (3.3) and Theorem 2.4 


X(a) 



E c »-x>) = E c * E (-i ) |D “ (0KS( “> 1 



Ahn Ahn QeSYT(A) 

Des(Q)eJ7 a 


E 

(_Xjl s ' x ' s '( a )l 

^2 Cx ■ \{Q e SYT(A) : Des(g) = S}\ 

SC[n-l] 


Ahn 

S£U a 



E 


■ | {beB: Des (b) = 5}| 

SC[n—1] 



S£U a 



E<- 

1 N| De s(b)xS(a)| 


b&B a 




for every a |= n, and conclude that B is a fine set for y. 

Conversely, assume that B is a fine set for x an d let B' be the disjoint 
union, taken over all partitions A b n, of c\ copies of SYT( A), e ndowed with 
the usual descent map Des : B' —> 2^ n ~ 1 f By Theorem 2.4 the set B' is 
also fine for y. Therefore, [6, Theorem 1.4] implies that the descent set is 
equidistributed over B and B' . This exactly means that (3.3) holds for every 
S C [n — 1] and the proof follows. □ 


Corollary 3.3. (see [6j Theorem 1.5]) Let B be a set endowed with a map 
Des : B — > 2^ n ~^ . The following are equivalent: 

(i) B is a fine set for some (non-virtual) character of & n . 

(ii) The quasisymmetric function Fq{x) := Yhb&B hi,Des(&)h) is symmetric 
and Schur-positive. 

(iii) There exist nonnegative integers a\, for A b n, such that 



z Bes (ft) 




J2 ^ DeS(Q) 

Ahn QeSYT(A) 


(3.4) 


where z = ( 21 ,..., z n -\). Moreover, if these conditions hold, then a\ = 
(F b (x), s\(x)) for each A bn. 
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3.2. Results for the hyperoctahedral group 

To state the £> n -analogues of these results, we will use the language of 
signed compositions and sets, explained in Section I Let a = (S,e) G S s (n) 
be a signed set, 7 G Comp B (n) a signed composition of n with set of partial 
sums S( 7) = {77, r 2 ,... ,r k } (so r k = n), and set r 0 = 0. The signed set a is 
called 7 -unimodal if S is unimodal with respect to the unsigned composition 
of n corresponding to 7. 

Definition 3.4. The weight wt 7 (cr) of a = (S,e) G £ B (?z) (equivalently, of 
the signed composition of n corresponding to a) with respect to 7 is defined 
as follows: 


• wt 7 (cr) := 0 if either a is not 7 -unimodal, or else for some index 1 < 
i < k the sign vector e G {—,+} n of a is not constant on the set 
{ 7 y_i + 1 ,... ,77}. 

• Otherwise we set 

wt 7 (cr) := (-l)l s ^WI +^(0 ( 3 . 5 ) 

where n 7 (cr) is the number of indices i for which the elements of {?y_ 1 + 
1 ,... ,77} are assigned the negative sign by (the sign vectors of) both 
a and 7. 

Given a character y of B n and a signed composition 7 G Comp B (n), we 
will write 7(7) for the value of y at the elements of B n of signed cycle type 
(ct,/3) b n, where a and fd are the partitions obtained by reordering the 
unbarred and barred parts of 7, respectively, in weakly decreasing order. 


Definition 3.5. Let y be a character of the hyperoctahedral group B n . A 
set B, endowed with a map sDes : B —> T, B (n), is said to be a fine set for y 

if 

xii) = E wt 7 (sDes(6)) (3.6) 

beB 

for every signed composition 7 of n. 


The main results of this paper are as follows. 


Theorem 3.6. Let y be a character of the hyperoctahedral group B n and B 
a set endowed with a map sDes : B —* £ s (n). Then B is a fine set for y if 
and only if 

ch (x) = ^ 2 F sDes{b) (x,y). (3.7) 

beB 
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In particular, the distribution of the signed descent set over B is uniquely 
determined by x- 

To state the third condition of the following corollary, given a = (S', e) G 
T B (n) we write z a = Y\_ ieS Zi, w h ere z % — x i (respectively, z, t = yf) if e* = + 
(respectively, £i = —). 

Corollary 3.7. Let B be a set endowed with a map sDes : B —* £ B (n). The 
following are equivalent: 

(i) B is a fine set for some (non-virtual) character of B n . 

(ii) The quasisymmetric function F B (x, y ) := F^^fx, y) is a Schur- 
positive element of A(x) ® A (y). 

(iii) There exist nonnegative integers a x ^, for (A,/i) b n, such that 

^Mb) = a Xtfl J2 z sBes(Q) . (3.8) 

b&B (A,/x)h n QeSYT(A,/i) 

Moreover, if these conditions hold, then a x ^ = (Fb(x, y), s x (x)s fl (y)) for each 
(A,/r) b n. 


4. Proofs 


This section proves Theorem 3T and deduces Corollary |3.7 from it. The 

as de¬ 


proof of one direction of Theorem 3.6 follows that of Theorem 3.2 


scribed in Section [3j it is based on a formula for the values of the irreducible 
characters of B n (see Theorem 4.1) which is analogous to Roichman’s rule 


for the irreducible characters of & n (Theorem 2.4). The challenge of finding 


analogues of Roichman’s rule for other (complex) reflection groups was im¬ 


plicitly raised in [9) page 45]. The proof of the other direction of Theorem 3.6 
requires the invertibility of a suitable weight matrix. 

4-1. Irreducible characters and bitableaux 

Given a composition a , denote by a + (respectively, a ~) the signed com¬ 
position obtained from a by considering its parts as unbarred (respectively, 
barred). The following result states that the set of standard Young bitableaux 
of shape (A ,ji) b n, endowed with the standard colored descent map defined 
in Section 2.2 is a fine set for the irreducible chararcter y A,/i of B n . 
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Theorem 4.1. For all bipartitions (A,//) b n and (cp/3) b n and for every 
permutation 7 of the parts of a + and (3~, 

X A,m O,/3) = Y wt 7 (sDes (Q)). (4.1) 

QeSYT(A,/t) 


In other words, SYT(A,/x) (with the associated map sDesj is a fine set for 
the irreducible character 


Proof. To compute x A,/i (a,,d) we expand the left-hand side of (2.6) as 


x 




<(«) m 

Pa( x ,v)pp( x ,y) = nM x )+PM)Il<p» 

i =1 3 =1 

= ^(- 1 ) l{j: ^=“ }l p l/ (a:)p € (x/) 

e,C 

where the sum ranges over all the vectors e— (ei,..., £t(a)) € { —, +}d") and 
( = (Cl,.. -,Cm) e {— ,+Y^\ and where v = v e ^ (respectively, £ = £ £, 0 
is the composition consisting of the parts a* of a with e* = + (respectively, 
£i = —) followed by the parts [fi of ft with Q = + (respectively, Q = —). 
Expressing each of p u (x) and pfijj) in the basis of Schur functions, according 
to (2.3), and comparing to (|2.6j) we get 


X A ’>,/3) = ^(-1) 
£>C 


|{i : Cj=-}| v A/,.e,C 


xW(£ 


e,C'| 


(4.2) 


where the summation and the compositions v £ X and £ £ b determined by the 
sign vectors e and C are as before. 


We now derive a similar formula for the right-hand side of (4.1), de¬ 
noted by ^ A,AI ( 7 )- Given sign vectors e = (£ 1 ,..., £e( a )) £ { —, +}d“) and 
C = (Ci, • • •, Ce(p)) G { —, + Y^\ denote by the signed composition whose 
underlying composition is that of 7 and whose parts are unbarred or barred, 
according to whether the corresponding parts of a and ft are assigned the + 
or — sign by £ and C, respectively. Denote by 7 ^ (respectively, the com¬ 
position obtained from by removing the barred (respectively, unbarred) 
parts and forgetting the bars. Express the set [n] as the disjoint union of 
contiguous segments whose cardinalities are the parts of 7, and denote by 


R £ jf (respectively, R^) the union of those segments which correspond to the 
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unbarred (respectively, barred) parts of 7 ^. The definitions of weight and 
signed descent set show that 


^(7) 


e>C 


■1) 


I(h0=-}| 



- 1 ) 


|Des(Q+)xS( 7 i.’ C )| 



- 1 ) 


|Des(Q-)xS( 7 I' C )| 


where the outer sum ranges over all sign vectors e, f as above and the inner 
sums range over all Young tableaux Q + of shape A with content R £ + and 
7 ^-unimodal descent set and over all Young tableau x Q~ of shape /i with 
content R e _^ and yi^-unimodal descent set. Theorem 
formula imply that 


2.4 


and the previous 




(4.3) 


= Et- 1 ) 10 ' 0 " ,l X A (7+’ C ).V(7' ,t ) 

e,C 

where the sum ranges over all sign vectors e G {—, +}d°0 and C, G { —, +}^. 
Since the values x( u ) °f an irreducible character x of a symmetric group do 
not depend on the ordering of the parts of the composition u, Equations (4.2) 


and (4.3) imply that y A ’ M (a,/3) = ^ A,M ( 7 )> as claimed by the theorem. 
Proposition 4.2. For all partitions X, y 


□ 


3\(x)s ll (y) = ^sDes (Q){x,y). 

QeSYT(A,/i) 

Proof. This statement follows from [25] Corollary 8 ] and Proposition |2.6| For 
a direct proof, it suffices to describe a bijection from the set of all pairs of 
semistandard Young tableaux S and T of shape A and p, respectively, to 
the set of all pairs ( Q,u ) of standard Young bitableau Q G SYT(A,p) and 
monomials u which appear in the expansion of F^^q^x, y), such that if 
( Q,u ) corresponds to ( S,T ) then x s y T = u. Given ( S,T ), we define Q by 
numbering the entries of T equal to 1, read from bottom to top and from 
left to right, with the first few positive integers 1 , 2 ,...; then the entries of 
S equal to 1 , read in the same fashion, with the next few positive integers; 
then the entries of T equal to 2 and so on, and set u = x s y T . We leave to 
the interested reader to verify that this induces a well defined map which has 
the claimed properties. □ 
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The following statement furnishes one direction of Theorem 3.6 


Proposition 4.3. Let x be a character of the group B n and let B be a set 
endowed with a map sDes : B —» S B (n). If 


ch(x) = ^F sDes{b) (x,y), (4.4) 

beB 

then B is a fine set for x- 

Proof. Let us express x — 52 a M )hn c \uX X,>1 as a linear combination of irre¬ 
ducible characters. We then have, by Proposition |4.2[ 

ch (x) = c a,m c 1 i (x a,m ) = c *,n s *( x ) s n(y) 

(A,/i)hn (A,/i)hn 

5 ^ c A,/x ^ ] P'sDes(Q){ x i t/)■ 

(A,/i)h n QeSYT(A,/i) 


Comparing with (4.4[) we get 


\{b e B : sDes (b) = a}\ = ^ c A)M ■ \{Q G SYT(A,/x) : sDes(Q) 

(A,/i)hn 




for every cr e S e (n). Using this equation and Theorem 4.1, we get 


C b/W A,M («,/?) = wt 7 (sDes (Q)) 

(A,/x)H n (A,Ai)hn QeSYT(A,/i) 

= wt 7 (cr) ^ c a, m ' |{<2 e SYT(A,/x) : sDes(Q) = <r}| 

cr££ s (n) (A,^t)hn 

= "^2 wt 7 (cr) • |{6 G B : sDes(5) = cr}| 

crGS s (n) 

= wt 7 (sPes(fe)) 
fees 

for every bipartition (a, (I) b n and every permutation 7 of the parts of a + 
and f3~ and the proof follows. □ 


For the proof of the other direction of Theorem 3.6 we need the following 


5 n -analogue of [ 6 , Theorem 1.4], which will be proved in Section 4.2 
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Theorem 4.4. If B is a fine set for a B n -character y, then the distribution 
of sDes over B is uniquely determined by y. 


Proof of Theorem 3. 6\ We only need to prove the converse of Proposition [L3 
Let B be a set which, endowed with a map sDes : B —)■ S B (n), is fine for a B n - 


character x■ We have to show that (4.4) holds. Express x = ^(a ^)\-n c \uX X,>1 
as a linear combination of irreducible characters. Let B' be the disjoint union, 
taken over all bipartitions (A,/r) b n, of C\ tfl copies of SYT(A,/r), endowed 
with the corresponding descent map sDes. By Theorem 4.1 the set B' is also 


fine for y. Theorem 4.4| now implies that sDes is equidistributed over B and 
B'. As shown at the beginning of the proof of Proposition 4.3[ 


ch(y) = ^2F sDes{b) (x,y ). 


b&B' 


Therefore, (4.4) holds as well. 


□ 


Finally, we deduce Corollary 3.7 from Theorem 3.6 
Proof of Corollary \3. 7j The equivalence 


l ll 


is a direct consequence of 


Theorem 3.6 The equivalence (ii) yy (iii) follows from Proposition 4.2 and 
the linear independence of the functions F a (x,y) for a E T, B (n). 


□ 


4-2. The weight matrix 

We now prove Theorem T4, the missing step in the proof of Theorem 3T 
using a H n -analogue of the Hadamard-type matrix A n from M- 


By Definition 3.4, a weight wt 7 (<r) is defined for any signed composition 
7 E Comp B (n) and signed set a E E B (n). The size of each of the sets 
Comp B (n) and E s (n) is d n := 2 • 3 n_1 . Fixing a linear order (to be specified 
later) on each of these sets, the weights can be arranged into a weight matrix 


I dn X- dn 


An. — (wt 7 (c)) 7 g Comp s( n y cr e s s (n) ^ {1;0,-1} C 

For a set B , endowed with a map sDes : B —)• E B (n), let v = v(B) be the 
column vector of length d n with entries 


Vfj = \{b E B : sDes(6) = cr}\ (Vcr E S' w (n)), 

where is in the specified linear order. For a character y of the hype- 

roctahedral group B n , let u = u( y) be the column vector of length d n with 
entries 




= xin) (Vy e Comp (n)), 


18 





















where Comp B (n) is in the specified order. Definition 3.5 can now be restated 
as follows: 

B is a fine set for x u (x) — A n ■ v(B). 


Theorem 4A is thus implied by the following result. 
Theorem 4.5. The matrix A n is invertible. In fact, 


det(A n )| = £[ 




7 EComp B {n) 


where, for a signed composition 7 G Comp (n) with part sizes 71,..., 7*, 


rm 


:= 2 k/2 • 


li 


2=1 


Theorem 4J3 will be proved using a recursive formula for A n and for an 
auxiliary matrix A n . 

Consider now the alphabet L = {0,1, *}, the set L n of all words of length 
n with letters from L, and the set Uf of all words w G L n in which the last 
letter is not *. Any signed set a = (S,e) G E B (n) is uniquely represented by 
a word w(a) = a± ■ ■ ■ a n G Lf , where 

{ 0, if i G S and e(i) = +; 

1, if i G S and e(i) = —; 

*, if 1 (f S. 

Recalling that we always have n G S, the map a 1 —> w(a ) is well defined and is 
clearly a bijection from S B (n) to Lf. The natural bijection from Comp B (u) 
to T, B (n) (see Section 2.1) yields a corresponding representation of signed 


compositions by words in Lf. 

By the above, we can consider the rows and columns of the weight matrix 
A n to be indexed by words w G L”. Let A n be the matrix obtained from 
A n by setting to zero all the entries indexed by (w, w') for which the initial 
sequence of *-s in w is longer than the initial sequence of *-s in w', namely: 
(A n ) WtW i = 0 if i(w) > i(w'), where 


i(w) := rninji | Wi 7 ^ *}. 
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This corresponds to a pair (7, cr) for which there is an element of S(a) before 
the end of the first part of the signed composition 7. 

For a word w £ L", let c(w) be the first letter in w which is not *: 


c(rc) . w p 

For a letter a' G {0,1}, let A“' be the matrix obtained from A n by setting to 
zero all the entries in columns indexed by words w' with c[w') 7 ^ a' . Clearly, 
A n = A® + Al. We use similar notation for A n . 


Definition 3.4 of the weight wt 7 (cr) implies the following recursive prop¬ 
erties of A n and A n : 

Lemma 4.6. For n > 1, letters a, a' G L and words w, w' G Uf, 


and 


(A 


n+l) aw,a'w' 


(A n+ i) c 


= 


n/in 1 * 



if a, a' G {0,1}; 

(-1 ) a<w ' ] {A n ) w , w/ , 

if a G {0,1}, a' = *; 

(^n )w,w' 1 

if a — *, a' G {0,1}; 

(-^ n)w,w ' j 

if a = a' = * 

(-l)“'(d n ) w y, 

if a, a' G {0,1}; 

(-1 ) a<w '\A n ) WjW ,, 

if a G {0,1}, a' = *; 

°C 

*/ a = *, a' G {0,1}; 

(A. n ) W:W i , 

if a = a' = *. 

l)a,a' = (-O'""' 

(Va, a' G {0,1}). 


Also, 


Specifying a linear order on L", we can write these recursions in matrix 
form. 


Corollary 4.7. Using the linear order 0 < 1 < * on L and the resulting 
lexicographic order on Lf, 


( A n A n A n \ 

A n+ i = | A n —A n A n — A n 1 
V-A° -A\ A n ) 


{n > 1 ), 
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1 A n A n 


Ar, 


A n -t-1 — [ A n A n A n A n 


(n > 1 ) 


0 0 


A r 


and 


1 1 

1 -1 r 


A\ — A 1 — 

Corollary 4.8. For n > l, 

det(Ai+i) = det(A n ) • det(-2 A n ) • det(A n + A n ) 

and 


det(Aj+i) = det(v4 n ) • det(-2A n ) • det(A n ), 


with 


det(Ai) = det(Ai) = —2. 

More generally, for every real a, 

det(«A n+ i + (1 - a)A n+1 ) = det(A n ) • det(-2A n ) • det (aA n + A n ) 
for n > 1, with 

det(aA 1 + (1 — oi)Ai) = —2. 

Proof. Using the fact that A n = A® PA], (and similarly for A n ), we can write 


f A Q -l A 1 A 0 - 1~ A 1 A 0 -i- 4 1 ' 

A n ->-i = I A G n + A\ -A° n — A\ A L° - Af 

— A 0 —A 1 4° 4- A 1 

jr *-'n -^-n Jr ^~n ' Jr ^~n / 


(■n > 1 ) 


and, more generally, 


( A® _i_ A 1 A 0 1 Al A 0 1 /I 1 ' 

^ /i n 1 r\- n r\- n l -r\- n 1 -^- n 

&A n+ i + (1 — a)A n+ i = ( A° n + A l n —A° n — A\ Aff — A n 

-aA°„ — aA^ A° n + A n/ 


0 n > 1 ), 


with 


01 A 1 "I - (1 — a)Ai — 


1 1 
1 -1 
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The set of nonzero columns of A° n is disjoint from the corresponding set for 
A^. We can thus perform elementary column operations using, separately, 
the columns of A° n and Af, to get 


det(aA n+ i + (1 - a)A n+ i) 


/A° n + A^ 
= det j A° n + A* 

\ 

(A^, + A 

= det j A L ° + A 


( 

'A° n + A l n 

A n + A n 

A n + 

det ( 

A n + A n 

- A °n ~ A n 

A ° n ~ 

\ 

S ~ aA °n 

-aA\ 

A n + 


0 

0 

\ 

-221° 

. - 2A\ 

~ 2A \ 


aA° n 

-aA l n 

aA n + A n + 

A1 J 


0 


0 


n w w 

l ~2A° n -2A 1 n 0 

aA° n — aAf aA° n + aA\ + 21° 



Note that in the last step we used only half of the columns of the middle 
—221° — 2A l n to annihilate the columns of — 2A*. The block triangular struc¬ 
ture of the resulting matrix implies that 

det(cu4 n+ i + (1 - a)A n+ i) = det(AJ • det(-2 A n ) • det (aA n + A n ) 


for n > 1, with 

det(aAi + (1 — Oi)A\) = —2. 

The special cases a = 1 and a = 0 give the results for A n and A ri 


D 


The following statement implies Theorem |4.5| and completes the proof of 
Theorem 13.61 


(4.5) 


Corollary 4.9. For all real numbers a and positive integers n, 
det(aA n + (1 — a)A n ) = — m 7 (a) 

7 SComp s (n) 


where, for a signed composition 7 G Comp s (n) with part sizes 71,..., 7fc, 

k 

m y (a) = 2 k/2 • (« 7 i + (1 - a)) ■ 

1=2 
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Proof. Setting 


A n (a) := det (uA n + (1 - a)A n ), 


Corollary 4.8 implies that, for a =4 — 1, 


A n+ 1 (a) = (2(a + l)) dn • A n (l ) 2 • A r 


a 


OL T 1 


for n > 1 and 

Ai(a) = -2. 

We have used here the fact that (—l) dn = 1, since d n = 2 ■ ?> n ~ l is even for 
all n > 1 . 

Consider now a signed composition 7 G Comp' B (n) with a corresponding 
word w G L*. It gives rise to three signed compositions of n+1, corresponding 
to the words Ota, lta, *w G L™ +1 . Using the notation m w (a ) instead of m 7 (a), 
it is clear that 


= 2 (fc+1)/2 • (a • 1 + (1 - a)) • 7* = 2 1/2 - m „(l), 

i=l 

whereas 

m*w(a) = 2 k/2 ■ (a ■ (71 + 1) + (1 - a)) • TT 7 * = (a + 1 ) • m w ( 01 V 


Therefore 


m 0w (a) ■ m lw (a) ■ m* w (a) = 2 (a + 1 ) ■ m w (l ) 2 ■ 


a 


OL T 1 


which implies that, at least for a > 0, both sides of equation (4.5) satisfy the 


same recursion (and, clearly, also the same initial conditions for n — 1). The 
two sides are therefore equal for all a > 0 and (being polynomials in a), are 
actually equal for all a. □ 

5. Knuth classes and involutions 

This section discusses irreducible B n -characters again, describes a differ¬ 


ent interpretation of Theorem 4.1 in terms of Knuth classes of type B and 
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confirms that the set of involutions in B n is fine for the character of the 
Gelfand model for this group. 

We first recall the definition of the natural analogue of the Robinson 
Schensted correspondence for the group B n m pages 145-146] [44]. Let 
w = (w(l),w(2),... ,w(n)) G B n and let (iu(ai), w(a 2 ),..., w(a k )) be the 
subsequence of unbarred elements of w. Applying Schensted’s correspon¬ 


dence to the two-line array ( , 1 N , 2 . , k , ) results in a pair 

" \ w(ai) w(a 2 ) ••• w(a k ) J 

(. P + ,Q + ) of Young tableaux of the same shape A b k. Similarly, from the 
subsequence of barred elements of w and the corresponding two-line array 
we get a pair (P~,Q~) of Young tableaux of the same shape fi\~ n — k. Then 
P B {w) = ( P + ,P ~) and Q B (w) = ( Q + ,Q ~) are standard Young bitableaux 
of shape (X, p) b n and the map which assigns the pair (P B (w),Q B (w)) to 
w is a bijection from the group B n to the set of pairs of standard Young 
bitableaux of the same shape and size n. 

The following two properties of this map are explicit in [44, Section 8] 
and implicit in [4IJ page 146], respectively. 


Proposition 5.1. For every vj € B n : 

(a) P B (w~ l ) = Q b (vj), 

(b) sDes(ta) = sDes(Q B (w)). 


A Knuth class of (type B and) shape (A,/i) is a set of the form {w G 
B n : P b (vj ) = T} for some fixed T G SYT(A,/i) (these Knuth classes should 
not be confused with the ones considered in the study of Kazdhan-Lusztig 
cells of type B ; see mm)- The first part of the following corollary, already 
discussed in Section [4[ is a restatement of Theorem 4.1 
follows from the first and Proposition 5.1 (b). 


The second part 


Corollary 5.2. For every bipartition (A,/r) b n, the following are fine sets 
for the irreducible B n -character y A ’K- 

(i) The set of standard Young bitableaux of shape (A,/r). 

(ii) All Knuth classes of shape (A, p). 


We recall that a Gelfand model for a group G is any representation of G 
which is equivalent to the multiplicity-free direct sum of its irreducible repre¬ 
sentations. The following proposition is a R n -analogue of the corresponding 
statement [3j Proposition 1.5] (see also [6[ Proposition 3.12 (iii)]) for & n . 
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Proposition 5.3. The set of involutions in B n , endowed with the standard 
signed descent map, is fine for the character of the Gelfand model of B n . 

Proof. Corollary |5.2| implies that the set of standard Young bitableaux of size 
n is fine for the character of the Gelfand model of B n . Moreover, as a direct 
consequence of Proposition |5.1[ the signed descent set is equidistributed over 
the set of standard Young bitableaux of size n and the set of involutions in 
B n . The proof follows from these two statements. □ 

Example 5.4. We confirm this statement for n — 2. The involutions in 
B 2 are the signed permutations (1,2), (2,1), (1,2), (1,2), (1,2) and (2,1). 
Therefore, 


^ ^ ^sDes(io) (*G y) 

w€lB2‘. W~ 1 =W 

= F( 2 )(x,y) +F (1)1) (x,y) + F {U) (x,y) + +F { 2 ) (x,y) + F {i}1) (x,y) 

= S( 2 )(x) + 5(1,1) (x) + s (1 )(x)s (1) (y) + 5(2) (y) + 5(1,1) (jj) 

= ch (x« 2 )’ 0 ) + x ((1 ’ 1} ’ 0) + x ((1)>(1)) + X (0 ’ (2)) + X (0 ’ (1 ’ 1)} ), 


where we have indexed functions F a (x,y) by signed compositions, rather 
than signed sets, and the second equality follows by direct computation or 


use of Proposition 4.2 


An inverse signed descent class in B n is a set of the form {w G B n : 
sDes(u> _1 ) = cr} for some a G E B (n). We postpone the definition of flag- 
major index to Section [6j where the representation of B n corresponding to 
the fine set (ii) in the following proposition will also be described. 


Proposition 5.5. The following subsets of B n are fine for some B n -characters: 

(i) All inverse signed descent classes. 

(ii) The set of elements of B n whose inverses have a given flag-major index. 


Proof. By Proposition |5.1[ the signed descent set sDes is fixed over an inverse 
Knutli class. It follows that inverse signed descent classes are unions of Knuth 
classes, which are fine sets by Corollary 5.2, and hence are fine sets as well. 
Morever, subsets of B n with fixed signed descent set have fixed flag-major 
index. As a result, the set of elements of B n whose inverses have a given 
flag-major index is a union of inverse signed descent classes and hence is fine, 
as a disjoint union of fine sets. □ 
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6. Coinvariant algebra and flag statistics 

This section provides a £> n -analogue of a result essentially due to Roich- 
man which describes explicitly fine sets for the characters of the action 
of ©„ on the homogeneous components of the coinvariant algebra of type A. 

Throughout this section we denote by P n the polynomial ring F[x 1 ,, x n ] 
in n variables over a field F of characteristic zero. The symmetric group © n 
acts on P n by permuting the variables. Let I n be the ideal of P n generated 
by the ©^-invariant (symmetric) polynomials with zero constant term. The 
group © n acts on the quotient ring P n /I n , known as the coinvariant algebra 
of ©„, and the resulting representation is isomorphic to the regular repre¬ 
sentation; see, for instance, m Section 3.6]. Since P n and I n are naturally 
graded by degree and the action of © n respects this grading, the coinvariant 
algebra is also graded by degree and & n acts on each homogeneous compo¬ 
nent. Denote by Xn,k the character of the ©^-action on the fc-th homogeneous 
component of P n /I n , for 0 < k < (™). 

Let w £ © n be a permutation. Recall [32] Sections 1.3-1.4] that inv(ry) 
denotes the number of inversions of w, Des(tc) its set of descents, and maj (w) 
its major index (the sum of the elements of Des(w)). Note also that inv(ry) = 
inv(u; _1 ) for w £ © n . 

Theorem 6.1. (|36] [6] Theorem 3.7]) For 0 < k < (”), each of the following 
subsets of & n , endowed with the standard descent map, is a fine set for the 
© n -character Xn,k : 

(i) {w £ & n : inv(u; _1 ) = k}; 

(ii) {w £ & n : maj(tc _1 ) = k}. 

Proof. The claim for (i) is a restatement of [361 Theorem 1], The claim for 
(ii) follows from the one for (i) by the fact m Theorem 1] that the number 
of w £ & n with Des(tc' 1 ) = S and inv(w) — k is equal to that of w £ © n 
with Des(w _1 ) = S and maj (w) = k for all S C [n — 1] and k. □ 

Consider now the group B n . It acts on the polynomial ring P n by permut¬ 
ing the variables x±,... , x n and flipping their signs. Let be the ideal of P n 
generated by the L> n -invariant polynomials (i.e., symmetric functions in the 
squares x\,... ,x^f) with zero constant term. The coinvariant algebra of B n 
is the quotient ring P n jiff. B n acts on each of its homogeneous components, 
and the resulting representation is isomorphic to the regular representation 
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of B n . It is graded by degree; denote by \nk ^ ie character of the Fraction 
on the fc-th homogeneous component of P n /I B , for 0 < k < n 2 . 

The flag-major index [4] of a signed permutation w G B n is defined as 


firiaj (w) = 2 • maj(iu) + bar (in), 

where inaj(m) is the sum of the elements of Des(in) (as defined in Section [272] ) 
and bar(in) is the number of indices i G [n\ for which w(i) is barred. The 
flag-major index of a standard Young bitableau Q of shape (A, /i) is de- 
hned as twice the sum of the elements of Des(Q) plus the size of /i, so that 
fmaj(in) = fmaj(Q B (in)) for every w G B n , in the notation of Section [ 5 } by 
Proposition 5.1 (b). The flag-inversion number of w G B n is defined as 

finv(in) = 2 ■ inv(in) + bar (in), 


where inv(in) is the number of inversions of the sequence (in(l), in( 2 ),..., w(n )) 
with respect to the total order ( 2 . 1 ) (this is a variant of the notion of flag- 
inversion number introduced in [2D] ; see also [TJ 22] and references therein). 
The equidistribution result [ 2T1 Theorem 1], mentioned in the proof of 


Theorem 6.1 follows from a theorem of Foata |19], which implies that inv and 
maj are equidistributed over the set of permutations of an ordered multiset 
(see, for instance, the discussion in [2Q\ Section 1]), by an application of 
the inclusion-exclusion principle. A similar argument yields the following 
analogous result for B n . 


Proposition 6.2. For every n, k and a G T, B (n), the number of w G B n 
with sDes(in -1 ) = a and finv(w) = k is equal to the number of w G B n with 
sDes(iu~ 1 ) = cr and friiaj (w) = k. 


Proof. Let us denote by f n ,k( cr) (respectively, g n ,k(&)) the number of w G B n 
with sDes(u; _1 ) = a and inv(u;) = k (respectively, maj (in) = k ). Since the 
function bar(iu) is constant within each class {in G B n : sDes(w -1 ) = cr}, we 
have to show that f n ,k (cr) = 9n,k{ a ) f° r all n, k and a G £ B (n). 

Fix n, k and a G S B (n). Let 7 G Comp B (n), with (absolute) parts 

7 1.. .. , 7 1 , be the signed composition corresponding to a. Consider the mul¬ 
tiset M. a obtained by breaking the set [n] into contiguous segments of lengths 

7 1.. .., 7 1 , replacing all the entries of a segment by copies of its smallest en¬ 
try, and barring them if the corresponding part of 7 is barred. For example, 
if 7 = (2,2, 2, 2) then M. a = {1,1, 3, 3, 5, 5, 7, 7}. We consider the ground 
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set of the multiset M. a totally ordered by (2.1). For r G £ B (n) we write 


r A a if the signed composition corresponding to r can be obtained from the 
one corresponding to a by replacing consecutive unbarred parts, or consec¬ 
utive barred parts, by their sum, while keeping the bar when present. For 
example, for 7 = ( 2 , 2 , 2 , 2 ), the signed sets r A a are those with signed 
compositions 7 , (4,2,2), (2,2,4) and (4,4). A little thought shows that the 
aforementioned equidistribution result of Foata [H], applied to the ordered 
multiset A4 a , implies that 


^ ^ /n,fc(' r ) y ' 9n,k (t') 


1 —<<7 


1 —<(T 


for every a G £ B (n). Using the principle of inclusion-exclusion, we conclude 
that / n ,fc(o r ) = 9n,k (f ) f° r every a G S B (n) and the proof follows. D 

The following statement is our /j„-analogue of Theorem |6.1| 

Theorem 6.3. For 0 < k < n 2 , each of the following subsets of B n , endowed 
with the standard signed descent map, is a fine set for the B n -character Xn,k : 

(i) {w e B n : hnv(t(; _1 ) = k}; 

(ii) {tc G B n : fmaj(uj _1 ) = k}. 


Proof. For every bipartition (A, /i) h n, the multiplicity of the irreducible 
character x A,M in \nk was shown by Stembridge (see [SSJ Theorem 5.3]) to 
be equal to the number of standard Young bitableaux of shape (A, / 1 ) and 
flag-major index k. This fact, Corollary 5.2 (i) and basic properties of the 
Robinson-Schensted correspondence for B n (see Proposition 5.1) imply the 
claim for (ii). The claim for (i) follows from that for (ii) by Proposition 6.2 


If 


Example 6.4. For n = k = 3 we have 


{w G B 3 : finv(tu) = 3} 

= {(I, 2, 3), (1, 2, 3), (1, 3,2), (2,1, 3), (1,3, 2), (2, 3,1), (3, 2,1)}, 


{w G B 3 : fmaj (w) = 3} 

= {(I, 2, 3), (1, 2, 3), (1, 3,2), (2,1, 3), (3,1, 2), (3, 2,1), (2, 3,1)}, 







{w £ 5 3 : finv(w *) = 3} 

= {(I, 2, 3), (1, 2, 3), (1, 3,2), (2,1, 3), (1,3, 2), (3,1,2), (3, 2,1)}, 
{w G B 3 : fmaj(w _1 ) = 3} 

= {(I, 2, 3), (1, 2, 3), (1, 3,2), (2,1, 3), (2,3,1), (3, 2,1), (3,1, 2)}. 

The signed descent compositions of the elements of either of the last two 
sets are (3), (1,1,1), (2,1), (1,2), (1,1,1), (1,1,1) and (1,1,1) and hence 


weB 3 

finv(w~ 1 )=3 


-^sDes(iy) (^5 V) 


weB 3 

fmaj(hw —1 )=3 

F(3){x,y) + 

F(i,i,i)(x,y) 


FsT)es(w) (*G y ) 


F( 2 ,i)(x, y) + F { i j2) (x,y) + 
(x,y) + F {ix i)(x,y) 


= s (3 )(y) + S( 2 )(x)s m (y) + s(i,i)0)s(i)(?/). 


The last equality follows by direct computation or use of Proposition 4.2 


Indeed, there are exactly three bitableaux of size 3 and flag-major index 3, 
namely 

)• 


(0 , 1 I 2 | 3 | ), ( 


1 


1 3 


Thus, by [i5j Theorem 5.3], 

ch (Xs-i) = ch (x (0 ’ (3)) 

= S (3 )(y) + S(i)(y)s(2)(a:) + S(i)(2/)s(i,i)(a:)- 


+ x (( 2 ),(l)) + X (( 1 , 1 ),( 1 ))J 


Remark 6.5. Another natural 7i n -analogue of the inversion number statistic 
inv : & n —» N is the length function Ib ■ B n —> N, in terms of the Coxeter 
generators of B n \ see m Section 8.1] for an explicit combinatorial description 
of this function. A formula of Adin, Postnikov and Roichman |2j Theorem 4] 
implies that, for every k, the subset {u> G B n : isiw) = k} of B n is a © n -hne 
set for the restriction of Xn,k to ©n- However, this set is not fine for any 
R n -character. Indeed, let us write Neg(cr) for the set of negative coordinates 
of the sign vector of a G T, B (n). Given a fine set B for some R n -character, 


condition (iii) of Corollary 3.7 implies that for any J C [n], the number of 
elements b G B with Neg(sDes(&)) = J depends only on the cardinality J 
and is divisible by (|”|). We leave it to the interested reader to verify that 
this restriction is violated in the case considered here. 


29 













7. Conjugacy classes 


This section reviews a result of Poirier [32], which implies that every 
conjugacy class in B n is a fine set, and discusses in detail two interesting 
examples of fine sets which are unions of conjugacy classes, namely the set 
of derangements in B n and the set of fc-roots of the identity element. 

Consider the alphabet A = {aq, aq, • • •, Hi, 2 / 2 , • ■ • }■ A necklace of length 
m over A is an equivalence class of words of length m over A, where two 
such words are equivalent if one is a cyclic shift of the other. A necklace 
is called primitive if the corresponding word (the choice of representative 
being irrelevant) is not equal to a power of a word of smaller length. Given a 
finite multiset A4 of primitive necklaces over A, the product of all variables 
appearing in the necklaces of A4 is called the evaluation of A4. The signed 
cycle type of A4 is the bipartition (a, f3) for which the parts of a (respectively, 
/3) are the lengths of the necklaces of A4 which contain an even (respectively, 
odd) number of y variables. We will denote by Lffp(x,y) the formal sum 
of the evaluations of all multisets of primitive necklaces of signed cycle type 
(a, j3) over A. 

The following result extends to B n analogous results for G n (see, for 
instance, the discussion in 


Section 2]). Combined with Theorem 3.6 it 
shows that each conjugacy class C a> p, endowed with the standard colored 
descent map, is a fine set for B n . 


Theorem 7.1. 


Theorem 16]) For every bipartition (cq/3) b n 


z 

™eC a ,/? 


F,D*tw)(x,y) = L^f(x,y) 


(7.1) 


Moreover, L^(x,y) is equal to the Frobenius characteristic of a representa¬ 
tion of B n . 

The representation of B n which appears in the theorem is a I? n -analogue 
of the Lie representation of given type of & n . We omit the definition, since 
this representation does not play any major role in this paper, and refer to 
[32 , Section 4] and references therein for more information. 


7. 1. Derangements 

The set of derangements in B n , being a union of conjugacy classes, is 
naturally a fine set. This section describes the corresponding representation 
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of B n and its decomposition as a direct sum of irreducible representations, 
and expresses its Frobenius characteristic in terms of another set of elements 
of B n , which may be called desarrangements of type B. These results are 
essentially £> n -analogues of results of Desarmenien and Wachs mm and 
of Reiner and Webb [341 Section 2] for the symmetric group, which will 
first be reviewed; they provided much of the motivation behind the present 
paper. A -analogue of the equidistribution m \W\ of the descent set 
among derangements and desarrangements in & n will be deduced. 

The set of derangements (elements without fixed points) in © n , denoted 
here by T> n , is a fine set for & n by Theorem 3.2 and [231 Theorem 3.6]. 
The corresponding ©^representation, first studied in [33] Section 2], can be 
described as follows. We denote by X n the set of words from the alphabet 
[n] with no repeated letters (known as injective words), partially ordered by 
setting u < v for u,v G X n if u is a subword of v. Thus, the empty word 
is the minimum element and the n\ permutations in G n are the maximal 
elements of X n . The poset X n is the face poset of a regular cell complex 
K, n whose faces are combinatorially isomorphic to simplices. The symmetric 
group © n acts naturally on X n and hence on the augmented cellular chain 
complex of fC n over C; see [33] and references therein for more explanation 
and for background on regular cell complexes. Following [33], we denote by 
Xn the character of the resulting representation on the top reduced homology 

^ n _!(/C n ,C). 

Using the Hopf trace formula and the shellability of /C n , it was shown in 
m Proposition 2.1] that 


P.2) 

k =0 

for every n (the reader who is unfamiliar with cellular homology may wish 
to consider this formula as the definition of y n ). 

We denote by E n the set of permutations (called desarrangements in [IT?] ) 
w G © n for which the smallest element of [n] \ Des(w) is even. The following 
theorem, which combines results of [15], [16] and m Section 2], shows that 
V n and {u^ 1 : w G £ n j are fine sets for the sign twist e n ® Xn of Xn and 
determines its decomposition as a direct sum of irreducible characters. 

Theorem 7.2. (pESUIE] |33l Section 2]) For every positive integer n, 
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ch-(<S n ® Xn) ^ ^ -^n,Des(tu) (*^) ^ ^ -^ri,Des(i(; — - 1 ) (^) 

weV n WdzBn 

= ^c x s x (x) 

Ah n 

where c\ is the number of standard Young tableaux P of shape X for which 
the smallest element of [n] \ Des(P) is even. In particular, the number of de¬ 
rangements w G T> n with Des(te) = S is equal to the number of permutations 
w G £ n with Des(w^ 1 ) = S for every S C [n — 1], 

We now turn attention to the hyperoctahedral group B n . We denote 
by the set of words from the alphabet £l n which contain no two letters 
with equal absolute values and partially order this set by the subword order. 
The poset Iff is the face poset of a regular cell complex /C,f whose faces are 
combinatorially isomorphic to simplices; see Figure [l] for the case n = 2. 
The group B n acts naturally on Iff and hence on the augmented cellular 
chain complex of ICff over C. We denote by if n the character of the resulting 
representation on the top reduced homology P n _i(/C)f, C). 


12 21 12 21 12 21 12 21 




Figure 1: The face poset of the cell complex /Cf. 

We denote by T>ff the set of derangements (elements without fixed points, 
when thought of as permutations of £l n ) in B n and by £ff the set of w G B n 
for which the maximum number k such that iv(l) > w(2 ) > • • • > w{k ) > 0 
is even, possibly equal to zero (where for a G 0„ we write a > 0 if a is 
unbarred). Finally, we denote by oj x the standard involution on A(x) < E > A(y) 
acting on the x variables. The following statement is our P n -analogue of 
Theorem 17.21 
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Theorem 7.3. For every positive integer n, 


u x ch('0„) 


^ ' -^sDes(u)) (•£, V) 'y ^ -^sDes(ui _1 ) (*^ 5 1/) 

w&VB w££B 

Y c a,/,sa (x)s ll (y) 


where c\^ is the number of standard Young bitableaux (P + , P~) of shape 
(A, /i) such that the largest number k for which 1, 2 ,..., k appear in the first 
column of P + is even (possibly equal to zero). 

In particular, the number of derangements w G V B with sDes(w) = a is 
equal to the number of signed permutations w G £ B with sDes(u> _1 ) = a for 
every a G Y B {n). 


Example 7.4. To illustrate the theorem, let us compute explicitly these 
expressions for oj x ch0 n ) when n = 2. The bitableaux which satisfy the 
condition in the statement of the theorem are 


, 0 ), ( 0 , 0 ), ( 0 , 00 ), (0 , 


)■ 


As a result, the third expression claimed for cv x ch(if n ) by the theorem gives 


U x ch{lf 2 ) = S ( i,i)(x) + S(i) (20(10) + s (2 0) + s (M 0). 
Moreover, the signed descent sets of the elements of 

V% = {( 2 , 1 ), ( 2 , 1 ), ( 2 , 1 ), ( 2 , 1 ), ( 1 , 2 )}, 

written as signed compositions, are (1,1), (1,1), (1,1), (I, I) and (2) and the 
same holds for those of {w -1 : w G £ B } = {(2,1), (1, 2), (1, 2), (2,1), (2,1)}. 
Hence, either of the first two expressions claimed for uj x ch(^ n ) gives 


Ux ch(0) = F {ltl) (x,y) + F {1>1) (x,y) + F {1>1) (x, y) + F$)(x,y) + F { i tl) (x,y). 


By direct computation or use of Proposition 4.2, one can verify that the two 
formulas for u x ch(z/> 2 ) are equivalent. 


The proof of part of Theorem IH will be based on the following propo¬ 
sition. The proof of the proposition is a direct analogue of the proofs of 
Propositions 2.1 and 2.2 in [MJ Section 2], 
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Proposition 7.5. For every positive integer n, 


= EM)”"" life, 




(7.3) 


k=0 


For every n > 2, 




B„ 


(7.4) 


Proof. Following arguments in [511 Section 2], we observe that the words of 
given rank k in if index the elements of a basis of the augmented cellular 
chain group Ck_i(/C,f, C). The group B n acts transitively on these words 
with stabilizer isomorphic to the Young subgroup (©i ) fc x B n _k- As a result, 
the character of B n actin g on Ck~\{]C C) is equal to the induced character 


1 x b k - Equation (]7.3|) then follows from the Hopf trace formula and 


the shellability (see [2HI Theorem 1.2]) of /C,f. Finally, using (7.3) we compute 
that 


v>„ = E(- 1 )"~‘ 1 if£ 


) k xB n -k 


k=0 


(-1)”1b„ + Ef - 1 )”' 11 it® 


(6i ) k xB n -k 


k=1 


= (- 1 )” 1 b„ + 


E(-i)”-* ltfri-. 


fc=i 


kBj i 


t 

(Si 


= (-i) n i Bn + (l®^i)t£ 


x Bn—1 


This verifies Equation (7.4). 


□ 


Proof of Theorem 7. 4 Applying the Frobenius characteristic to Equation (7.4) 


and using Lemma 245 gives 

ch(^n) = si(x,r/) ch(-0 n -i) + (-l) n s„(x) 


and hence 


uj x ch('f n ) = si(x,y) ■o; a: ch(^ Tl _i) + (~l) n e n {x) (7.5) 


34 










for n > 2, where si(x,y) = Si(x) + Si(y). One can check that ipi is the sign 
character of B i, so that ch(^q) = o; x ch(^i) = si(y). By this observation and 
an easy induction argument, the recurrence ( 7.5[ ) and Pieri’s rule imply that 

UJxCh^n) = Y c x, l iSx(x)s fl (y), 

(A ,/i)hn 


(7.6) 


where C\ 4l is as in the statement of Theorem 7.3 Furthermore, using Propo¬ 
sition |T2] and basic properties of the Robinson-Schensted correspondence of 
type B (see Proposition |5.1[ ) we get 

Y c a,/^a (x)s ll (y) 

(A,/t)bn 


'y C A,/i 'y ] -^sDes (Q){x,y) 
(A ,/Phn QeSYT(A,/i) 


E E -^sDes(Q) ( x i y) 

Pe syt'(a,aO QsSyt(a,/7) 

^ ^ -^sDes(ui) ( x i ?/) j 


(7.7) 


where SYT'(A, /^) stands for the set of standard Young bitableaux P = 
(P + , P _ ) such that the largest number k for which 1, 2,..., k appear in the 
first column of P + is even (possibly equal to zero). 

Finally, we set 


D n( X >y) ■= Y F ^es(w)(x,y) 

W&VB 


for n > 1. Since P® is a union of conjugacy classes, Theorem 
that 

D n( x ,y) = Y L «A x ’y) 

where the sum ranges over all bipartitions (cq/3) h n such that a has no part 
equal to one. We now recall (see [2H Section 3] and references therein) that 
there is an evaluation and length preserving bijection from the set of words 
on the alphabet A to the set of multisets of primitive necklaces on A. Thus, 
just as in the symmetric group case (see the proof of j23) Theorem 8.1]), 
summing L^ j3 (x, y) over all (cq/3) h n and considering the number of parts 
of a equal to one we get 


7.1 implies 


si(x,y) n = Y s *( x ) D n-k( x ,y) 
k =o 
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for n > 0, where s 0 (a;) = D^(x,y) = 1. Equivalently, we have 


D*{x,y) = ^2 (-l) fe e fe (x)si(x,?/) 


n—k 


(7.8) 


fc =0 


for n > 1. This formula and (7.5) show that 

D n(x,y) = WjChfV'n) 


(7.9) 


for every n > 1. The proof follows by combining Equations (7.6), (7.7) and 

ra. □ 


Remark 7.6. The last statement of Theorem 7.2 was originally proven in 
na. A bijective proof was later provided in [16]. An analogous proof of the 
last statement of Theorem |7.3| should be possible. 


7.2. k-roots 

Given a positive integer k, we will denote by r n ^{w) the number of k- 
roots of w G ©„ (meaning, elements u G & n with u k = w) and by r^ k (w) the 
number of fc-roots of w G B n . Clearly, r n ^ and k are class functions on & n 
and B n , respectively. It was shown by Scharf [38] S§] that these functions 
are actually (non-virtual) characters of & n and B n . The first statement in 
the next theorem follows from [37, Theorem 1.1] and its proof; the second 
statement follows from the first and Theorem 13.21 

Theorem 7.7. ([37]) The set {w G & n : w k = e} of all k-roots of the identity 
element in & n is a fine set for the character r n ^. Equivalently, 


ch(r n} k) ^ ^ ^ri,D e s(w) (*^) 

w£&n'-w k =e 


(7.10) 


for all positive integers n, k. 


The following theorem gives a partial answer to [37], Question 3.4]. It 
is expected that this theorem can be extended to the (possibly more com¬ 
plicated) case of even positive integers k, which is left open in the present 
writing (the case k — 2 follows from Proposition 
the character of the Gelfand model of B, 


see 


5.3 


since r^ 2 is equal to 


p. 58]). 


n, 2 
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Theorem 7.8. The set {ru G B n : w k = e} of all k-roots of the identity 
element in B n is a fine set for the character r^ k for every odd positive integer 
k and all n > 1. Equivalently, 

c H r n,k) = F sd es (w)(x,y) (7.11) 

w£B n : w k =e 

for all positive integers n , k with k odd. 


Example 7.9. For n = k = 3, the fc-roots of the identity element in B n 
are (1, 2, 3), (2,3,1), (3,1,2), (2,3,1), (2,3,1), (2,3,1), (3,1,2), (3,1,2) and 
(3,1, 2). Computing the signed descent compositions of these signed permu¬ 
tations and applying Theorem |7.8, we find that 


ch(rj 3 ) = F (s)(x,y) + F {2 ,i)(x,y) + F {h2) (x,y) + F [% i)(x,y) + 
F (i, 2 ){x,y) + ( x , y) + 2F(j 1 j)(x, y) + Tpjjp 


The multiset of these signed compositions coincides with the multiset of 
signed descent compositions of the standard Young bitableaux of shapes 
((3),0), ((2,1),0), ((1), (2)) and ((1), (1,1)). Thus, we may deduce from 
Proposition |4.2| that 

ch 0 f 3 ) = s (3)0r) + s (2l i)(aO + s ( i)(a;)s (2 )(y) + s {1) (x)s (ltl) {y). 


This yields the decomposition r® 3 = ^1(3),0) _|_ ^(( 2 ,i), 0 ) _|_ ^.((i),( 2 ) ^((i),(i,i))_ 


The proof of Theorem 74 


follows the computation of ch(r n fc) in 
Exercise 7.69 (c)]. We will write L^ 


as 

presented in the solution to (33j, Exercise 7.69 (c)]. We will write L%(x, y) for 
the formal sum of the evaluations of all primitive necklaces of length n over 
the alphabet A having an even number of y variables (i.e. for the function 
La,six, y) when a = (n) and /3 = 0). We will also write L n (x,y) for the 
corresponding formal sum with no restriction on the number of y variables. 
We will first establish the following lemma. 


Lemma 7.10. For every odd positive integer k 


d\k n> 1 n> 1 


1 

2 n 


(PnKnM&y))^ + (Pn/Mfay))™) t% 


where x n = (xf, x %,...), y n = (y™,yf,...) and ( n,k ) denotes the greatest 
common divisor of n and k. 
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Proof. The definition of the functions L d (x,y ) and L^(x,y) as generating 
functions of necklaces implies that L d (x, y ) = (. L d (x , y) + Ld(x, —y))/2. Since 

E E v") = E (](p W*’ vP*' *” < 7 - 12 ) 

d|fc n>l ra>l 

follows from [43| Equation (7.216)], it suffices to verify that 

= E »))'”*’*”■ < 7 - 13 ) 

d|fc n>l n>l 


We sketch the proof of this equation, which is similar to that of (7.12). Let 
us denote by n : {1,2,...} —> 7L the number theoretic Mobius function. 
The following well known (see, for instance, [241 Equation 2.2] or [43] Exer¬ 
cise 7.89 (a)]) formula 


!.(>) = -PM(va(x)) mli 

m z — J 


d | m 


can be proved by a simple Mobius inversion argument. Replacing x by 
fx n l m 1 — y n / m ^ i n this equation and assuming that m is odd gives 


d | m d | m 

whenever n is divisible by m. Applying Mobius inversion, we get 

(Pn/m( X .y)) m = J2 dL ^ X " /d ’-V" ,d 'l 

d | m 

whenever m is odd and n is divisible by m. Since k is assumed to be odd, 
we may replace m by (n, k) in the last equation to get 


(Pn/(n,k)( X ^y)) 


(n,k) 


Y,Y. dL ^ xn '' d -y"'^ 

d\k d\n 


for every positive integer n. Multiplying by t n /n and summing over for all 
n > 1 we get (7.13) and the proof follows. □ 
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Proof of Theorem 7.8. From the defining equation (2.5) of the Frobenius 
characteristic map we have 


ch (r* k ) = 


1 


2 n n\ ^ 

w£B r 


^ r nA W )Pw(x,y) = 


1 


2 n n! 


Y Pu k ( x ,y)- 


u£B n 


As in the solution to [121 Exercise 7.69 (c)] we observe that the kth power of a 
positive m-cycle in B n is a product of (m, k ) disjoint positive cycles of length 
m/(m,k ) and, since k is odd, the analogous statement holds for negative 
cycles. An application of the exponential formula [43, Corollary 5.1.9] then 
yields 


, \ t n 

Y Ch ( r n,k) tn = exp Y{(Pn/(n,k)( X ’y)) (n,k) + (Pn/(n,k )( X ’^ 

n> 1 


n> 0 


(7.14) 

We now denote by G n: k(x, y) the right-hand side of (7.11) and compute its 
generating function as follows. Since k is odd, a signed permutation w G B n 
is a fc-root of the identity element if and only if the cycle decomposition of 
w involves only positive cycles whose lengths divide k. Thus, it follows from 
Theorem O that 


Y G n,k( X ’y) tn = Y L «^ X ’ y ^ tH 

n> 0 a 


where the sum ranges over all integer partitions a such that every part of a 
divides k. The definition of L^^x^y) in terms of multisets of necklaces and 
that of plethysm of symmetric functions in turn imply that 

Y Gr i*i x ,y)t n = I \h[ L a(tx,ty)] (7.15) 

n >0 d\k 


where h{x) = h n (x) is the formal sum of all complete homogeneous 

symmetric functions h n (x). The basic formula log h(x) = log D[j>i 1/(1 — 
xf) = ^)n>i Pn{x)/n and (7.15) then imply that 


Y,Gn.k(x,y)r = 


(7.16) 


n>0 


d\k n >1 


Comparing (7.14) to (|7.16|) and using Lemma 7.10 gives ch(r^ fc ) = G nj k(x,y ) 
for all n > 0 and the proof follows. □ 
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8 . Arc permutations 

Arc permutations, originally introduced in the study of triangulations |5] , 
have interesting combinatorial properties. For instance, they can be charac¬ 
terized by pattern avoidance, they carry interesting graph and poset struc¬ 
tures, as well as an affine Weyl group action, and afford well factorized un¬ 
signed and signed enumeration formulas ini- Two type B extensions were 
introduced in |T 8 ]. This section reviews the relevant definitions and shows 
that one of these extensions is a foie set for an ©^-character, while the other 
is a fine set for a R n -character. 

A permutation w G © n is said to be an arc permutation if, for every 
i G [n], the set {u>(l), u>(2),... ,w(i)} is an interval in 7L n (where the letter 
n is identified with zero). For example, (2,1,5,3,4) is an arc permutation 
in ©5 but (2,1, 5, 6 , 3,4) is not an arc permutation in © 6 . The set of arc 
permutations in © n is denoted by A n . 

As will be explained in the sequel, the following theorem can be deduced 
from results of Elizalde and Roichman Id- This theorem will be extended 
to type B in this section. Let V n -\ be an ( n — l)-dimensional vector space 
over C, on which ©„_i acts by permuting coordinates. 

Theorem 8.1. For n > 2, the set A n is fine for the character of the induced 
representation f\V n -\ t 6n of the exterior algebra ofV n -\ from © ra _i to © n . 

Equivalently, J2 w ga„ F n,T>es{w)( x ) = ch(y) where 

n—l n—1 n—2 

X = + X (1 ’> + 25> <mb -‘> + £ x<* A1 "“ _ ’>. 

k=l k =2 k =2 

Two type B extensions of the concept of an arc permutation, suggested 
in [18], can be described as follows. Let us identify with Z 2n by associating 
i G Q n with n + i G Z 2n (and thus h G fl n with 0 G Z 2ri .) for every i G 
[n]. A signed permutation w G B n is said to be a B-arc permutation if 
{w(i),w(i +1),..., w(n)} is an interval in Z 2n for every i G [n]. For instance, 
(2, 3,1, 5,4) is a B- arc permutation in B§. The set of R-arc permutations in 
B n will be denoted by . Note that \A^ \ = n2 n . 

We also say that w G B n is a signed arc permutation if, for each i G 

{2,...,n — l}, 

• the set (|n?(l)|,..., \ w{i — 1 ) 1 } is an interval in Z n ; and 
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• w(i) is unbarred if |u>(i)| — 1 G {|w(l)|,..., | w(i — 1)|} and barred if 
|iu(i)| + 1 G (|u>(l)|,..., | w(i — 1)|} (with addition in Z n ). 


For instance, (3, 2,4,1) is a signed arc permutation in h> 4 . The set of signed 
arc permutations in B n will be denoted by A s n . Since there is no restriction 
on the signs of w(l) and w(n), the number of signed arc permutations in B n 
satisfies \A s n \ = A\A n \ = n2 n for n> 2. 


The main results of this section state that A% and A * are fine sets for 
characters of & n and B n , respectively, which are described explicitly. We 
recall from Section 2.2 that the descent set of w G B n is defined as Des(u;) = 
{i G [n — 1] : w(i) > r w(i + 1)}, and that its signed descent set sDes(tc) is 
given by Definition 2.2| The group B n _ 1 acts on the vector space W-i by 
permuting coordinates and switching their signs. 


Theorem 8.2. For every n > 2, 


^ ^ ^n,T)es(w) i.x') 


n —1 


ch 2 ^ 


X 


(k,r 


b 

I r 


S„-i 


k= 1 


k=l 


Theorem 8.3. For every n > 2, the set A s n is fine for the B n -character 
induced from the exterior algebra f\\7 n ~\. Equivalently, 

t n—1 

^ x (D,a n b 

k =0 



Example 8.4. We have 

A* = A s 2 = {(1, 2), (1,2), (1, 2), (I, 2), (2,1), (2,1), (2,1), (2,1)}. 

Thus, for n — 2, Theorem |8.2| states that 

4 F 2 ,e>{x) + AF 2y {i}fx) = 4 s^fx) + 45(1,1) (x) 

= Ch (2x (1) tl: +2(x' u > + X (2) )). 

and Theorem 18.31 states that 


F( 2 ) (x, y) + 2F(i i) (x, y) + 2F (I; i } {x, y) + F {2) (x, y) + F (M) {x, y) + F (IJ) (x, y) 
= s ( 2 )(x) + 5(2) (y) + spp) (x) + S(i,i) (y) + 2s w (x)s w (y) 

= ch ((x^m + x m*) tg), 

where we have indexed the functions F a (x 1 y) by the corresponding signed 
descent compositions. 
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For the proof of Theorems |8.1| and |8.2| we need one more definition. A per¬ 
mutation w € & n is called left-unimodal (respectively, right-unimodal ) if the 
set (w(l), w(2 ),..., w(n)} (respectively, {w(i),w(i + 1),..., w(n)}) is an in¬ 
terval in Z for every i G [n] . Clearly, all left-unimodal and all right-unimodal 
permutations are arc permutations. The set of left-unimodal permutations 
in & n will be denoted by C n , and that of right-unimodal permutations by 

Tl n - 

Proposition 8.5. For n > 1, 

E F n ,Des(w) ( x ) = E F nVes(w)(x) = ch | E ^ 

w£Cn w£lZ„ \ k =1 

Proof. As explained in the proofs of Propositions 4 and 6 in in. from the 
definition of left and right unimodality we get 



^ ^ ^n,Des(w) (^) ^ ^ Tn, Des(ui) (*^) ^ ^ 

w£C„ w£TZ„ SC[n— 1 ] 

A simple application of Proposition [276 shows that 


E F n,s(x) = E s (k,l"~*)(x) 

SC[ra-l] k=1 


and the proof follows. 


oi 


Proof of Theorem FJ_. We may assume that n > 4. Then m Theorem 5] 
may be rephrased as stating that 


n —2 


E F n.Ties(w) (X) = Ch ( 

W E-4-n \ (£n U7?-n ) 


k =2 


The second statement in the theorem follows from this equation, Proposi¬ 
tion |8.5| and the fact that C n D 7 l n consists of the identity permutation and 
its reverse. The equivalence to the first statement is explained in the proof 
of [TTj, Theorem 6]. □ 
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Proof of Theorem 8.2. 


The proposed equation may be rewritten as 


^ ' -fn,Des(ui) (*^) 
w£A% 


n— 1 


n —2 


= ch (n + 2)(x (n) + X (in) ) + (n + 4) 




(fc,i n - fe ) 


+ 




X 


(fc,2,l” 


k=2 


k =2 


To prove this equation, and in view of Theorem 8.1 and Proposition 8.5 


it suffices to find a subset A n of for which there exist an n-to-one descent 
preserving map from A n to the set 7 Z n of right-unimodal permutations and 
a two-to-one descent preserving map from A% \ A n to the set A n of arc 
permutations. We will verify these properties when A n is the set of w G A% 


such that w(i) = 1 for some i G [n\. For the first property, we observe that 
A n is the disjoint union of the sets A n> k, consisting of the elements w G A„ 
for which (rc(l), w(2),... ,w(n)) is a permutation of {A; + 1,..., n, 1,..., k}, 
for 1 < k < n and that there is a natural descent preserving bijection from 
each A n to P. n . 

For the second property, we note that A n ■— Af \ A n is the set of w G A^ 
for which (iu(l), w(2), ..., w(n )) is a permutation of {k + 1,..., n, 1,..., k} 
for some k G [n] and consider the map / : A n —> A n which simply forgets 
the bars, meaning that f(w) = (|w(l)|, |w(2)|, ..., |w(n)|) for w G A n . This 
map is clearly descent preserving. We leave to the reader to verify that the 
preimages of a given arc permutation (cti, 02 ,..., a n ) G A n under / are 


• (1, 02, ..., a n ) and (1, 02,..., a n ), if a\ = 1 , and 

• (ai, b 2 ,.. ■ , bn) and (ai, b 2 ,..., b n ) otherwise, where b t = o.,; if a* > a\ 
and bi = a t if < ai, for i > 2 . 


This shows that the map / is also two-to-one. 


□' 


Remark 8 . 6 . Using the reasoning in Remark 6.5, it can be verified that Af 
is not a fine set for any R„-character for n > 3. 

Even though A*, is not a union of Knuth classes of type B. the Robinson- 


Schensted correspondence of type B will be used in the proof of Theorem 8.3 


We recall from Section 2.2 that we think of signed permutations w G B r 
written in the form (iy(l), w(2 ),..., w(n)). 


as 
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Proof of Theorem 8.3. 


^ ^ FsT)es(w){pCi V') 
w&A s n 


The proposed equation may be rewritten as 

n— 1 

s (n)(x) + S(in)(y) + ^ S( fci i)(a;)s( 1 n-fc-i)(j/) + 

k =1 

n —1 n —1 

g (fc-i)(^) g (2,i"- fc ~ i )(y) + ^ g(fc)(x)s(ir»~fc)(?/). 

k= 1 /c=l 


By Corollary 3.7 (or Proposition 4.2), it suffices to show that the distri¬ 
bution of sDes over A s n is equal to its distribution over all standard Young 
bitablcaux of the following multiset of shapes: 


(i) shapes ((n),0) and (0, (l n )) with multiplicity one; 

(ii) all shapes (( k , 1), (l ,l_fc_1 )) and (fn — k — 1), (2, l fc_1 )), for 1 <k< n — 1, 
with multiplicity one; and 

(iii) all shapes (( k ), (l n ^^)), for 1 < k < n — 1, with multiplicity two. 


Given our discussion of the Robinson-Schensted correspondence for B n 
in Section [5] (and, in particular, since the map Q B (-) preserves the signed 
descent set), it suffices to show that the restriction of Q B {-) to A s n induces 
a two-to-one map from a subset A s n of A s n to the set of standard Young 
bitableaux of shapes of type (iii) and a bijection from A s n \ A s n to the set of 
standard Young bitableaux of shapes of types (i) and (ii). 

Let e G B n be the identity permutation and w Q G B n be defined by 
w Q (i) = n — i + 1 for i G [n\. We note that A s n is invariant under right 
multiplication by w 0 . For k G [n— 1], we denote by A n ,k the set consisting of 
all signed permutations which are shuffles of the sequences (k, k — 1, ..., 1) 
and (k + 1,... ,n). Then A n ,k Y A s n and A nt kW 0 consists of all shuffles of 
(n, n — 1,..., k + 1) with (1,2 ,..., 7c) . Clearly, A n ,k and A n ^w 0 are disjoint 
for k G [n — 1] and A n ^, U A n ^w 0 and A n j U A n jW a are disjoint for i 7^ j. We 
choose A s n to be the disjoint union 


71—1 

:= [J (•'V* U An,kWo)- 

k =1 


For w G A n ,k, we note that Q B {w ) is the standard Young bitableau of shape 
(( n—k ), (l fc )) whose entries in (l fc ) are the positions of the barred letters in w. 
As a result, Q B {-) induces a bijection from A n ^ to SYT((n — k), (l fc )). Simi¬ 
larly, Q B (-) induces a bijection from A n ^w 0 to SYT((fc), (l n_fc )). Thus, Q B (-) 
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induces a two-to-one map from A s n to the set of standard Young bitableaux 
of shapes of type (iii). 

Finally, we need to show that Q B (-) induces a bijection from A s n \ A^ 
to the set of all standard Young bitableaux of shapes of types (i) and (ii). 
Let us denote by C ntk the set of all shuffles of (n — k + 1,... ,n,l) with 
(n — k, n — k — 1,.. ., 2 ) and note that C n ^ is a subset of A s n . Denoting by c 
the positive cycle (12 • • • n)(l 2 ■ • • n) and letting 

k- 1 

Bn,k := LJ ^ C n,k, 

r=0 

we may express A s n as a disjoint union 

n—1 n— 1 

A s n ^A s n = {e,w} U |J B n , k U [J B n , k w 0 . 

k= 1 k= 1 

Clearly, Q B (e ) is the only standard Young bitableau of shape ((n), 0 ) and 
Q B {w 0 ) is the only standard Young bitableau of shape (0, (l n )). We leave to 
the reader to verify that Q B (-) induces bijective maps from B ri)k and B n ^w 0 
to the set of standard Young bitableaux of shape (( k , 1), (l n-fc-1 )) and ((n — 
k, 1), (2, l fc-1 )), respectively. This completes the proof. □ 


9. Remarks 


Character formulas for the hyperoctahedral group were studied in this 
paper. A generalization to the corresponding Iwahori-Hecke algebra would 
be most desirable. 


Problem 9.1. Find an appropriate g-analogue of Definition 3.5 which will 
provide character formulas for the Iwahori-Hecke algebra of type B. 


Most of the results in this paper may be naturally generalized to the 
wreath product group G(r,n ) := Z r l S n of r-colored permutations, where r 
is an arbitrary positive integer (this paper has focused on the special case 
of G(2, n) = B n ). For example, the F> n -analogue of the fundamental qua- 
sisymmetric functions, considered here, was introduced and studied in the 
general setting of r-colored permutations by Poirier [32]. However, it should 
be noted that generalizations to G(r,n ) are not always straightforward. The 
following problem is challenging. 
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Problem 9.2. Generalize the concept of signed arc permutations and The¬ 
orem 


8.3 to the groups G(r,n ) 


The situation is more complicated for general complex reflection groups. 

Problem 9.3. Generalize the setting and results of this paper to the classical 
complex reflection groups G(r,p,n). 


For example, the generalization of Theorems A7 and A8 is related to 
the problem of characterizing the finite groups G for which the function 
r*. : G —>• N, assigning to w G G the number of fc-roots of w, is a non-virtual 
character; see, for instance, m Chapter 4] [39]. 

The group of even signed permutations D n = (7(2, 2, n) is of special in¬ 
terest. 

Another possible direction is to extend the setting of this paper to other 
(not necessarily classical or finite) Coxeter groups. 

Problem 9.4. Generalize the concept of fine set to arbitrary Coxeter groups. 


This objective may require a different approach. For example, we be¬ 
lieve that the results in Section [6] may be generalized to classical complex 
reflection groups. This hope is based on the current good understanding of 
the flag-major index and its role in the combinatorics of the coinvariant and 
diagonal invariant algebras of classical complex reflection groups; see, for in¬ 
stance, ra- Unfortunately, exceptional Weyl groups are still mysterious and 
the problem of finding a “correct” flag-major index on these groups is widely 
open. 
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